Derivation of the Iteration Algorithm for the Modified Pseudo-Inverse Model for Associative Memory from the Consideration of the Energy Function

The associative memory has been one of the most extensively studied artificial neural networks [1]. The limit storage capacity for the optimal weight matrix is known to be 2N where N is the number of the neurons and the learning algorithm is given by the perceptron learning [2]. The Hebbian rule gives the storage capacity of about 0.14N [3], [4]. The pseudo-inverse type of algorithm yields the capacity of N [5], [6]. The pseudo-inverse model is also obtained by the iteration algorithm which can be traced back to the iterative algorithm proposed by Ben-Israel [7]. This algorithm, when used for a finite number of iterations, gives a model somewhere between the Hebbian type and the pseudo-inverse type. For example, Dotsenko et al. [8] obtained the replica-symmetric free energy for a finite number of iterations from which the perfomance of the model was estimated. Wimbauer et al. [9] discussed the relationship between this type of iteration and the process of ‘unlearning .’ Kanter and Sompolinsky [6] investigated in detail the properties of the pseudo-inverse model without the self couplings (the Hamiltonian version of the pseudo-inverse model) by obtaining the mean-field solution by the replica method and showed by simulation that it has a larger basin of attraction for the embedded patterns than the model having the self couplings. The present letter shows that simple consideration of the energy of the network leads to the iteration algorithm which is identical to the above iteration except for